Policy is in the details: Using external validity to help policy makers

Criminologists lament that their voices are seldom heard in the rooms of policy. Although all interest groups suffer similar distress, we criminologists do have a point. Much criminal justice policy is what some of my colleagues call “quackery” (Latessa, Cullen, and Gendreau, 2002); it is devoid of not only evidence but also of logic. Some approaches to crime and justice receive little policy support, despite their supporting evidence. And as important, some evidence-based policies are misinterpreted, misapplied, and misused. Criminologists may not be able to prevent dumb ideas from being applied, and they may have difficulty getting policy makers to adopt evidence-based practices. Still, criminologists do have some ability to reduce the misinterpretation, misapplication, and misuse of their evidence. They can be clear about how policy works. A lack of clarity muddles and obfuscates criminological policy discourse. Clarity about how a policy works is every bit as important as clarity about statistical methods, research designs, and measures of constructs. Science demands that studies be described in sufficient detail that they can be replicated. A study that cannot be replicated becomes the subject of grave suspicion, as is exemplified by cold fusion (Taubes, 1993). Similarly, criminologists who desire to influence policy must describe the policies they study with sufficient clarity that a policy maker could implement it, should that the policy be desirable, or avoid it if the policy is undesirable. Clarity is less a matter of style than of providing the right information. What is this information? This is the subject of this essay. I argue that policy relevance is related to the external validity. Not external validity as viewed by Berk and MacDonald (2010, this issue), but as viewed by the people who may consider applying a study’s findings. Criteria for external validity (Shadish, Cook, and Campbell, 2002) provide a useful list of information authors should provide policy makers. I apply these criteria to the study by Berk and MacDonald to illustrate how information about these criteria can aid